This invention relates generally to a three-dimensional shape measurement technique, and more particularly, to a method and apparatus for measuring a three-dimensional shape based on phase shifting method and to a computer program product for use with a three-dimensional shape measurement apparatus.
Up to now, a variety of techniques for measuring a three-dimensional shape of an object have been proposed. A grating pattern projection method, introducing a striped scanning, as one of such techniques, is first explained based on reference (1) entitled: xe2x80x9cGrating Projection System for Profiling with the Aid of Fringe Scanning Method xe2x80x9d Journal of Precision Engineering (JSPE), vol. 55, No. 10, pp. 1817-1822, 1989.
FIG. 11 shows schematic diagram of a three-dimensional shape measurement device disclosed in the reference (1). Referring to FIG. 11, a light pattern having a sinusoidal luminance pattern is projected from a light source 101 on an object 100 through a sinusoidal grating 102 having a gray scale values printed thereon sinusoidally. A point 104 on the object 100 is scanned by a camera 103 which outputs an image data. If a coordinate value of the point 104 on an image taken by the camera 103 are denoted x and a luminance value on the coordinate x is denoted I(x), the luminance value I(x) is given by a following equation (1)
I(x)=a0(x)+A(x)cos(xcfx86+xcex1(x))xe2x80x83xe2x80x83(1)
where a0(x) is a bias component and xcfx86, xcex1(x) denote the phase.
The image is captured by the camera 103, each time after shifting the grating 102, by a length equal to 1/N of the wavelength of a printed sinusoidal pattern N times, along an axis u, with the object 100 remaining stationary.
The image appears as if the sinusoidal light pattern projected on the object 100 is proceeding 2xcfx80/N radian each time. Assuming that the phase xcfx86 is shifted from 0 radian up to 2(Nxe2x88x921)xcfx80/N radian where N is a positive integer, with an increment of 2xcfx80/N radian each time, the luminance value Ik(x) at a point x obtained for the kth shifting (0xe2x89xa6kxe2x89xa6N) is given by a following equation (2)
                                          I            k                    ⁡                      (            x            )                          =                                            a              0                        ⁡                          (              x              )                                +                                    A              ⁡                              (                x                )                                      ⁢                          cos              ⁡                              (                                                                            2                      ⁢                                              xe2x80x83                                            ⁢                      π                      ⁢                                              xe2x80x83                                            ⁢                      k                                        N                                    +                                      α                    ⁡                                          (                      x                      )                                                                      )                                                                        (        2        )            
The phase xcex1(x) is a phase value at a point x in an image photographed for k=0. The point 104 is present on a half line originating from the coordinate x on the camera screen to pass through a lens center. The Point 104 as viewed from the light source 101 is present on a plane of the sinusoidal grating 102 determined by a straight line with the phase xcex1(x) and the light source 101. Therefore, if a point of intersection between the straight line and the plane is found, it may be seen that the three-dimensional coordinate value of the point of intersection is that of the point 104.
By introducing two new coefficients xcex11(x) and B1(x), shown by a following equation (3), the above equation (2) can be rewritten to a following equation (4):                                                                                           a                  1                                ⁡                                  (                  x                  )                                            =                                                A                  ⁡                                      (                    x                    )                                                  ⁢                cos                ⁢                                  xe2x80x83                                ⁢                                  α                  ⁡                                      (                    x                    )                                                                                                                                                            b                  1                                ⁡                                  (                  x                  )                                            =                                                -                                      A                    ⁡                                          (                      x                      )                                                                      ⁢                sin                ⁢                                  xe2x80x83                                ⁢                                  α                  ⁡                                      (                    x                    )                                                                                                          (        3        )                                                      I            k                    ⁡                      (            x            )                          =                                            a              0                        ⁡                          (              x              )                                +                                                    a                1                            ⁡                              (                x                )                                      ⁢            cos            ⁢                          xe2x80x83                        ⁢                                          2                ⁢                                  xe2x80x83                                ⁢                π                ⁢                                  xe2x80x83                                ⁢                k                            N                                +                                                    b                1                            ⁡                              (                x                )                                      ⁢            sin            ⁢                          xe2x80x83                        ⁢                                          2                ⁢                                  xe2x80x83                                ⁢                π                ⁢                                  xe2x80x83                                ⁢                k                            N                                                          (        4        )            
a1(x) and b1(x) may be found by a following equation (5), using luminance values I0(x), . . . INxe2x88x921(x) at the point x obtained on Nth image-capture operations, whilst the phase xcex1(x) may be found by a following equation (6)                                                                                           a                  1                                ⁡                                  (                  x                  )                                            =                                                1                  N                                ⁢                                                      ∑                                          k                      =                      0                                                              N                      -                      1                                                        ⁢                                      xe2x80x83                                    ⁢                                                            I                      k                                        ⁢                                          cos                      ⁡                                              (                                                                              2                            ⁢                                                          xe2x80x83                                                        ⁢                            π                            ⁢                                                          xe2x80x83                                                        ⁢                            k                                                    N                                                )                                                                                                                                                                                                      b                  1                                ⁡                                  (                  x                  )                                            =                                                1                  N                                ⁢                                                      ∑                                          k                      =                      0                                                              N                      -                      1                                                        ⁢                                      xe2x80x83                                    ⁢                                                            I                      k                                        ⁢                                          sin                      ⁡                                              (                                                                              2                            ⁢                                                          xe2x80x83                                                        ⁢                            π                            ⁢                                                          xe2x80x83                                                        ⁢                            k                                                    N                                                )                                                                                                                                                    (        5        )                                          α          ⁡                      (            x            )                          =                              tan                          -              1                                ⁢                      xe2x80x83                    ⁢                                    -                                                b                  1                                ⁡                                  (                  x                  )                                                                                    a                1                            ⁡                              (                x                )                                                                        (        6        )            
The phase value of the object 100 on the image data is obtained by executing the above-described phase calculations for each pixel on the image taken by the camera 103.
Meanwhile, xcex1(x) obtained from the above equation (6) is unexceptionally wrapped (folded) between xe2x88x92xcfx80 and xcfx80, as may be seen from a fact that calculation is made using an arctangent function tanxe2x88x921( ). The result is that the phase xcex1(x) as found exhibits indefiniteness corresponding to an integer number times 2xcfx80, such that, in this state, a three-dimensional shape of the object 100 cannot be found.
By projecting a sinusoidal pattern composed of a period on the entire object 100, the Phase xcex1(x) can be uniquely determined. Since a narrow phase value from xe2x88x92xcfx80 to xcfx80 is allocated at this time to the entire object 100, a high measurement accuracy cannot be realized.
For this reason, a method is adopted to improve a measurement accuracy at the cost of phase uniqueness in which a domain of an initial phase is enlarged and a sinusoidal pattern of plural periods is projected on the object 100. FIG. 12 shows a Phase image 105 which is an example of the phase as found in each pixel within an image date captured by the camera 103. In FIG. 12, a phase taking a value from xe2x88x92xcfx80 to xcfx80 is allocated to black to white.
If, for example, a plane is measured, as shown in FIG. 12, there is obtained non-continuous phase, so that it is necessary to determine a relative phase value within an image by suitable techniques to convert non-continuous phases into continuous values. Also, since an absolute phase value cannot be directly obtained, the absolute phase needs to be determined by some means or other.
One of conventional phase connection technique for converting the Phase wrapped to between xe2x88x92xcfx80 and xcfx80 into a continuous phase is described in, for example, reference (2) of T. R. Judge and P. J. Bryanstonxe2x80x94Cross, entitled xe2x80x9cA review of Phase Unwrapping Techniques in Fringe Analysisxe2x80x9d, Optics and Lasers in Engineering, Vol. 21, pp. 199-239, 1994. The phase fringe counting/scanning approach technique, which is the simplest one of the phase connection methods, is hereinafter explained based on the description on pp. 211-212 of the reference (2).
This technique is constituted by following steps:
(1) Noise is removed by applying a filter having a spatial spreading or an low-pass filter (LPF) exploiting Fast Fourier Transform (FFT) to an input image. From the noise-eliminated image, the phase is calculated using e.g., the equation (6).
(2) A threshold value processing is executed for phase difference between neighbouring pixels to find a non-continuous phase boundary (changes in value from xcfx80 to xe2x88x92xcfx80).
(3) The image is scanned from row to row in a horizontal direction, and 2xcfx80 is added or subtracted each time a non-continuous phase boundary is traversed to find a continuous phase value between neighboring pixels.
(4) Phase values, continuous from row to row are compared in a vertical direction, and further converted to phase values which are continuous throughout the entire image.
In general, the phase connection method, inclusive of the above-described technique, suffers a drawback that satisfactory results cannot be obtained unless the supposition holds that a surface of the object 100 is sufficiently smooth, no significant changes in values are contained in the measured phase values, and that noise is only small. For this reason, the phase connection processing can pose a significant problem in automatic measurement.
The phase connection processing simply converts the phase value, wrapped from xe2x88x92xcfx80 to xcfx80, so that the phase value will be continuous in its entirety, without obtaining the phase value as the absolute value. Therefore, there is left indefiniteness equal to xcfx80 multiplied by an integer number in the phase value.
In the above reference (1) entitled xe2x80x9cMethod for Projecting Grating Pattern Introducing Striped Scanningxe2x80x9d, there is stated the converting method from an absolute phase value to a three-dimensional coordinate value, however, there is not described the method of determining an absolute phase value.
By the same author as that of the reference (1), a method including the technique of determining the absolute phase value is described in a reference (3) entitled xe2x80x9cGrating Projection System for Profiling with the Aid of Fringe Scanning Method, Second Reportxe2x80x94Exploitation of Sinusoidal Gratingxe2x80x9d, Journal of Precision Engineering, vol. 58, No. 7, pp. 1173-1178, 1992.
In the reference (3), automatic discrimination is rendered possible on an image by thickening one of the gratings by a factor of 1.5 as compared to the remaining gratings to remove indefiniteness from the phase value.
The method of finding three-dimensional coordinate values from the acquired absolute phase value is explained by referring to FIG. 13. A light pattern is projected from a light source 101, placed at a coordinate position (X0, Z0) to the object 100 through a grating 102 placed at a separation of i in the optical axis direction nxe2x88x92 where a symbol xe2x88x92 denotes a vector. On the grating 102 is printed a sinusoidal wave at a pitch p such that the initial phase will be 0 on the optical axis nxe2x88x92. It is assumed that the angle between the optical axis nxe2x88x92 and the X-axis is xcex8.
It is also assumed that light having an absolute phase a is illuminated on a point (X, Y) which is the point 104, as has been viewed by a camera 103 having a focal length f. It is likewise assumed that the point 104 is projected on a coordinate X, on a screen 106 in the camera 103. An angle of view xcex3 from the camera 103 to the point 104 may be found by a following equation (7);                               tan          ⁢                      xe2x80x83                    ⁢          γ                =                  x          f                                    (        7        )            
A following equation (8) also holds.                               tan          ⁢                      xe2x80x83                    ⁢          γ                =                  X          Z                                    (        8        )            
Moreover, it is assumed that the light pattern having an absolute value a is illuminated at an angle xcex8+xcex2, with respect to the X-axis. The angle xcex2 can be obtained, by exploiting a grating period p and a distance I between the light source 101 and the grating 102, by a following equation (9):                               tan          ⁢                      xe2x80x83                    ⁢          β                =                              α            ·            p                                2            ⁢                          xe2x80x83                        ⁢            π            ⁢                          xe2x80x83                        ⁢            l                                              (        9        )            
The angle of field xcex8+xcex2 from the light source 101 to the point 104, with the X-axis as a reference, is given by a following equation (10):                               tan          ⁡                      (                          θ              +              β                        )                          =                              Z            -                          Z              0                                            X            -                          X              0                                                          (        10        )            
With use of the above equations (7) to (10), the coordinate (X, Z) of the point 104 may be found by an equation (11) from the Position x on a screen 107 being imaged, absolute phase xcex1 and the constitution of device:                               Z          =                      f            ·                                                                                (                                                                  2                        ⁢                                                  xe2x80x83                                                ⁢                        π                        ⁢                                                  xe2x80x83                                                ⁢                        l                        ⁢                                                  xe2x80x83                                                ⁢                        tan                        ⁢                                                  xe2x80x83                                                ⁢                        θ                                            +                                              α                        ⁢                                                  xe2x80x83                                                ⁢                        p                                                              )                                    ⁢                                      X                    0                                                  -                                                      (                                                                  2                        ⁢                                                  xe2x80x83                                                ⁢                        π                        ⁢                                                  xe2x80x83                                                ⁢                        l                                            -                                              α                        ⁢                                                  xe2x80x83                                                ⁢                        p                        ⁢                                                  xe2x80x83                                                ⁢                        tan                        ⁢                                                  xe2x80x83                                                ⁢                        θ                                                              )                                    ⁢                                      Z                    0                                                                                                                    (                                                                  2                        ⁢                                                  xe2x80x83                                                ⁢                        π                        ⁢                                                  xe2x80x83                                                ⁢                        l                        ⁢                                                  xe2x80x83                                                ⁢                        tan                        ⁢                                                  xe2x80x83                                                ⁢                        θ                                            +                                              α                        ⁢                                                  xe2x80x83                                                ⁢                        p                                                              )                                    ⁢                  x                                -                                                      (                                                                  2                        ⁢                                                  xe2x80x83                                                ⁢                        π                        ⁢                                                  xe2x80x83                                                ⁢                        l                                            -                                              α                        ⁢                                                  xe2x80x83                                                ⁢                        p                        ⁢                                                  xe2x80x83                                                ⁢                        tan                        ⁢                                                  xe2x80x83                                                ⁢                        θ                                                              )                                    ⁢                  f                                                                    ⁢                  
                ⁢                  X          =                      Z            ·                          x              f                                                                    (          11          )                ⁢                  xe2x80x83                    
In another reference (4) (JP Patent Kokai A-11-118443), there is proposed a technique by a triangular wave which employs only summation, subtraction, multiplication and division instead of the arctangent function tanxe2x88x921( ) which, while having high precision performance similar to the sinusoidal projection as in the reference (1), is high in calculational cost in phase calculation, as shown by the equation (6). Since this technique is basically the variation of the above reference (1), there are similarly required the phase connection method and the absolute phase determination. Similarly to the above technique of the reference (1), these can be executed by a technique similar to that of the above references (2) and (3).
For further details, the disclosure of the reference (1) to (4) mentioned hereinabove are incorporated herein by reference thereto.
The conventional method described in the reference (1) and (4) (shape measuring apparatus), mainly suffers from following five drawbacks.
(1) Because of different optical axes of the light source 101 and the camera 103, there exists an area which can be viewed from the camera but which is not illuminated by a sinusoidal pattern, such that a three-dimensional shape cannot be measured in this area.
(2) There is an area that cannot be viewed from the camera 103 such that a three-dimensional shape cannot be measured in this area.
(3) An absolute value of a phase is difficult to find automatically such that Phase values different by an integer number factor of 2xcfx80 cannot be discriminated. That is, an absolute three-dimensional shape cannot be obtained.
(4) It is difficult in general to automatically execute the technique of continuously connecting phase values that is wrapped from xe2x88x92xcfx80 to xcfx80.
(5) Due to various problems relating to light projectors, it is difficult to construct an apparatus in which sinusoidal and triangular waves can be handled in an ideal fashion as described in the above-cited references.
It is noted that above mentioned problem (3) is concerned with determination of an absolute phase. In determining the absolute phase, there is a following relationship of trade-off between an automatic processing and an absolute three-dimensional shape measurement.
(a) The sinusoidal fringe on the grating 102 is marked such as by changing its width to permit automatic detection.
(b) The position of the object 100 is determined by e.g., a manual operation so that a suitable point on the object for measurement 100 will be at a pre-set measurement position. The corresponding position is designated, such as by a manual operation and command, for the measured phase value, to determine the absolute phase.
(c) A suitable site on the surface of the object 100 is measured by e.g., a manual operation to get it reflected on a measured phase value.
(d) An absolute three-dimensional shape is given up. The offset of the phase value is automatically determined by a suitable technique.
The above method (a) is described in the reference (3). With this method, it is difficult to discriminate if the sinusoidal luminance distribution as observed with the camera 103 is really wide or if simply it looks as if it is wide due to the three-dimensional shape of the object 100 being measured.
The above (b) and (c) are methods which give up automatic processing to use a manual operation thus appreciably lowering the range of application.
The above (d) is automated in that a suitable offset is determined so that the phase value at a mid position of the image will be comprised within the range of xe2x88x92xcfx80 to xcfx80. This method gives up absolute three-dimensional shape measurement.
Referring to FIG. 14, the problem encountered when an absolute phase value is unknown. It is assumed that, by a process up to a phase calculation and a phase connection that phase values xcex1 and xcex1+2xcfx80 are obtained in a direction of a viewing line A111 and in the direction of a viewing line B112 from a camera 103, respectively, with phase values changing smoothly between two viewing lines.
A direction 113, along which the absolute phase xcex1 is projected, through a grating 102 from a light source 101, and directions 114, 115 and 116, along which absolute phases xcex1+2xcfx80, xcex1+4xcfx80 and xcex1+6xcfx80 are projected similarly, are extended, as shown in FIG. 14.
If an absolute phase values are not known from FIG. 14, an object surface as viewed by the camera 103 is not clear as to whether this object surface is an object surfaces A108, an object surface B109 or an object surface C110.
It is noted that appropriately setting a phase offset is synonymous with appropriately selecting one of the object surfaces A108, B109 or C110. As may be readily seen from FIG. 14, there are occasions where, as a result of the selection, measured values of the surface inclination or length give results different from real values. Since an article being measured is made up of a variety of surfaces, each of these surfaces is similarly transformed to give a generally distorted shape of the measured object. Although an automatic measurement is made possible by using this method, an absolute three-dimensional shape is not obtained, leading to only a narrow range of application.
The above-mentioned problems (1) and (2) are relevant to the problem of a dead angle. For solving the problems (1) and (2), necessary numbers of light sources and cameras are required to install, respectively.
However, the technique of the above-mentioned reference material (3) by itself is not sufficient to acquire an absolute three-dimensional shape automatically.
As a result, plural measured results of a three-dimensional shape, as obtained from plural light sources or camera sets, give independent distortions. Thus, if light sources or cameras are provided additionally, the results of automatic measurements cannot be unified or synthesized, such that some form of a manual operations is required.
The problem (4) is relevant to the problem of phase connection. As discussed in the above-mentioned reference (2), if it is desired for the conventional phase grating counting/scanning technique to output good results, it is presupposed that a clear boundary be obtained when the phase value skips from xe2x88x92xcfx80 to xcfx80.
However, in reality, it is a frequent occurrence that redundant boundaries are presented or necessary boundaries disappear under the effect of noise. Similar problems are encountered with other conventional techniques. For example, in the above-mentioned reference (2), it is stated that a low performance of phase connection processing is a problem remained to be solved in realising an automatic three-dimensional shape measurement.
As for the problem (5), which is related with the actual equipment structure, it is not easy to generate the waveform that can be theoretically handled easily in connection with machining accuracy. In the above-cited reference (1), it is stated that the phase value xcex1 as found by the equation (6) from a pattern corresponding to the spontaneously blurred Ronky grating is not vitally changed from the phase value as found from the correct sinusoidal pattern. However, deviation from e.g. a sinusoidal wave significantly affects the phase value xcex1, as calculated by the above equation (6), as stated in reference (5) by Katherine Creath, xe2x80x9cComparison of Phase-Measurement Algorithmsxe2x80x9d, SPIE Vol. 680, Surface Characterization and Testing, pp. 19-28 (1986), the entire disclosure whereof is incorporated herein by reference thereto. If, for example, the hardware cost is to be reduced or measurement with higher precision is aimed at, such a technique needs to be realized which is able to compensate the deviation from a theoretical sinusoidal or triangular wave.
In view of the above-mentioned problems, it is an object of the present invention to combine one or more light sources and cameras in a three-dimensional shape measurement based on a phase shift method to provide a method and apparatus (1) to overcome the problem of a dead angle, (2) to realize a decision of absolute phase values, and (3) to enable an automatic execution of reliable phase connection processing.
It is another object of the present invention (4) to provide a method and apparatus less costly and which can assure measurement performance of higher precision.
According to a first aspect of the present invention, there is provided a method for measuring a three-dimensional shape comprising the steps of:
(a) projecting a light pattern having spatially striped luminance distribution on an object being measured, from light projection means, while phase-shifting the light pattern;
(b) scanning the object illuminated with the light pattern while phase-shifting the light pattern from at least two different directions by cameras to output first and second image data;
(c) deriving first and second initial phase images from a series of the first and second image data, respectively;
(d) finding a set of corresponding three-dimensional coordinate positions from an initial phase of a target pixel, in the first initial phase image, based, on a position of the light pattern projection and a first scanning position of the first image;
(e) finding a set of pixel positions corresponding to the set of three-dimensional coordinate positions in the second initial chase image based on a position of the light pattern projection and a second scanning position of the second image, having reference to initial phases of the respective pixel positions, comparing the initial phases to an initial phase of the target pixel in the first initial phase image, and verifying whether or not a corresponding pixel position having a same initial phase can be uniquely determined;
(f) finding a three-dimensional coordinate position based on the uniquely determined pixel position to determine an absolute value of an initial phase of a corresponding pixel in the first and second initial phase images;
(g) repeating the steps (d) to (f) for entire pixels of the first initial phase image;
(h) converting the first and second initial phase images into first and second absolute phase images in reference to the absolute value of an initial phase of the light pattern; and
(i) finding a three-dimensional coordinate position of the object being measured in each pixel, based on the absolute phase in each pixel of the first and second absolute phase images, the projection position of the light pattern and the scanning positions of the first and second images.
According to a second aspect, in the method of the first step, the step (f) of finding the three-dimensional coordinate position from the initial phase image comprises sub-steps of:
(1) finding, based on the projecting position of the light pattern and on the first scanning position, a set of candidates of first three-dimensional coordinate positions, from the initial phase of the target pixel in the first initial phase picture;
(2) finding a set of pixel positions corresponding to a set of the first three-dimensional coordinate positions, in the second initial phase image, based on the second scanning position, comparing, in reference to an initial phase of each of the pixel positions, the resulting initial phase to the initial phase of the target pixel in the initial phase picture, to verify whether or not a position of a corresponding pixel having a same initial phase can be uniquely determined;
(3) finding a set of candidates of corresponding second three-dimensional coordinate positions from an initial phase of a uniquely determined pixel in the second initial phase image based on the projecting position of the light pattern and the second scanning position;
(4) finding a set of pixel positions corresponding to the set of second three-dimensional coordinate positions in the initial phase image based on the first scanning position, and comparing, in reference to the initial phase of the respective pixel positions in the second initial phase image, the initial phase to the initial phase of the uniquely determined pixel in the second initial phase image, to verify whether or not a position of the corresponding pixel having a same initial phase can be uniquely determined;
(5) finding a three-dimensional coordinate position from the pixel position uniquely determined in the steps (2), and (4) to determine an absolute value of the initial phase of a corresponding pixel in the first and second initial phase images, respectively; and
(6) repeating the steps (1) to (5) for the entire pixels of the first initial phase image.
As a third aspect, the object being measured is scanned by cameras from three or more different directions.
In a fourth aspect, there is provided a method for measuring a three-dimensional shape comprising the steps of:
(a) projecting first and second light patterns, having spatially striped luminance distribution, by light projection means, from two different directions, while phase-shifting the light pattern;
(b) illuminating the first light pattern, while phase shifting the first light pattern, for scanning an object being measured by a first camera, illuminating the second light pattern, while phase shifting the second light pattern, for scanning the object being measured by a second camera;
(c) deriving first and second initial phase images from the first and second images captured by first and second cameras with the first and second light patterns being projected;
(d) finding a set of corresponding three-dimensional coordinate positions from the initial phase of the target pixel in the first initial phase image, based on the projection position of the first light pattern and the scanning position;
(e) having reference to a set of possible phase values, from a projection position of the second light patten and a corresponding set of three-dimensional coordinate position candidates, and comparing an initial phase of the target pixel in the second initial phase image to verify whether or not a same initial phase can be determined uniquely;
(f) determining absolute values of the initial phases of the first and second patterns from the uniquely determined three-dimensional coordinate positions;
(g) repeating the steps of from (d) to (f) for the entire pixels of the initial phase image;
(h) converting the first and second initial phase images into images of first and second absolute phases in reference to the absolute values of the initial phases of the light patterns, respectively; and
(i) finding a three-dimensional coordinate position in each pixel of the object being measured, based on the absolute phase in each pixel of the first and second absolute phase images, projection positions of the two light patterns and the image photographing positions.
As a fifth aspect, in the method of the fourth aspect, the step (f) determining the absolute position of the initial phase from an initial phase image comprises sub-steps of:
(1) finding a first set of corresponding three-dimensional coordinate position candidates from the initial phase of the target pixel of the first initial phase image, based on the projection position of the first light pattern and the scanning position;
(2) having reference to a set of possible phase values, from an illumination position of the second light pattern and the first set of three-dimensional coordinate position candidates, and comparing the phase values with the initial value of the target pixel in the second initial phase image to verify whether or not the coincident initial phase can be determined uniquely;
(3) finding a second set of corresponding three-dimensional coordinate position candidates from the initial phases of the target pixel in the second initial phase image based on the projection position of the second light pattern and the scanning position;
(4) having reference to a set of possible phase values, from an illumination position of the first light pattern and the second set of three-dimensional coordinate position candidates, and comparing the phase values with the initial value of the target pixel in the first initial phase image to verify whether or not the coincident initial phase can be determined uniquely;
(5) determining absolute values of the initial phases of the two light patterns from the three-dimensional coordinate positions uniquely determined from the steps (2) and (4); and
(6) repeating the steps (1) to (5) for the entire pixels of the first initial phase image.
As a sixth aspect, the method in the fourth and fifth aspect, further comprises the step of:
projecting a light pattern, having spatially striped luminance distribution, by light projection means, on the object being measured, from three or more different directions, while phase-shifting the light pattern.
As a seventh aspect, there is provided a method for measuring for measuring a three-dimensional shape comprising the steps of:
(a) projecting first and second light patterns, having spatially striped luminance distribution, by light projection means, from two different directions, on an object being measured, while phase-shifting the light patterns;
(b) illuminating the first light pattern, while phase shifting the first light pattern, to scan an image of the object by first and second cameras, followed by illuminating the second light pattern, while phase shifting the second light pattern, to scan the object by the first and second cameras from two different directions in a similar manner;
(c) deriving first and second initial phase images, respectively, from series of the first and second image data captured by the first and second cameras with the first light pattern from two directions;
(d) deriving third and fourth initial phase images from third and fourth images, respectively, captured by the first and second cameras with the second light pattern, from respective two directions;
(e) finding a first set of corresponding first three-dimensional coordinate position candidates, from the initial phase of the target pixel in the first initial phase image obtained from the first scanning position by the first light pattern, based on the projecting position of the first light pattern and the first scanning position;
(f) finding a set of corresponding pixel positions of the first set of the three-dimensional coordinate position candidates in a fourth initial phase image obtained from the second scanning position by the second light pattern, based on the second scanning position, to find a second set of three-dimensional coordinate position candidates from the initial phases of the set of corresponding pixels;
(g) comparing the sets of the first and second three-dimensional coordinate position candidates to verify whether or not overlapping coordinate points can be determined uniquely;
(h) determining absolute values of the initial phases of the first and second light patterns from the uniquely determined three-dimensional coordinate positions;
(i) repeating the steps of from (e) to (h) for the entire pixels of the first initial phase image;
(j) converting the first to fourth initial phase images to the first to fourth absolute phase images, in reference to the absolute values of the initial phases of the light patterns, respectively;
(k) finding a three-dimensional coordinate position of the objet being measured in each pixel based on the absolute phase in each pixel of the first to fourth absolute phase images, projecting positions of the first and second light patterns and on the first and second scanning positions.
As an eighth aspect, in the method of the seventh aspect, the step (h) of determining the absolute value of the initial phase from the initial phase image comprises sub-steps of:
(1) finding a first set of corresponding three-dimensional coordinate position candidates from the initial phase of the target pixel in the first initial phase image obtained from the first scanning position by the first light pattern, based on the projection position of the first light pattern and the first scanning position;
(2) finding a set of corresponding pixel positions of the set of three-dimensional coordinate position candidates in a fourth initial phase image obtained from the second scanning position by the second light pattern, based on the second scanning position, to find a set of second three-dimensional coordinate position candidates from the initial phase of the set of corresponding pixels;
(3) comparing the first and second sets of three-dimensional coordinate position candidates to verify whether or not an overlapping coordinate point can be determined uniquely;
(4) finding a third set of corresponding coordinate position candidates from the initial phase of a pixel corresponding to the uniquely determined three-dimensional coordinate position, based on the second scanning position;
(5) finding a set of corresponding pixel positions of the set of the third three-dimensional coordinate position candidates in the first initial phase image obtained from the second scanning positions, by the second light pattern, based on the first scanning position, to find a fourth set of the coordinate position candidates from the initial phase of the set of corresponding pixels;
(6) comparing the sets of the third and fourth three-dimensional coordinate position candidates to verify whether or not an overlapping coordinate point can be determined uniquely;
(7) determining absolute values of the initial phases of the first and second light patterns from the uniquely determined three-dimensional coordinate position; and
(8) repeating the steps (1) to (7) for the entire pixels of the initial phase image.
As a ninth aspect, in the method of the seventh aspect, the step (h) of determining an absolute value of the initial phase from an initial phase image comprises sub-steps of:
(1) a step of finding a set of corresponding three-dimensional coordinate position candidates from the initial phase of the target pixel in the first initial phase image, based on the projection position of the first light pattern and on the first scanning position;
(2) finding a set of pixel positions corresponding to the set of three-dimensional coordinate positions, in the second initial phase image obtained from the second scanning position by the first light pattern, based on the second scanning position, and comparing, in reference to respective initial phases, the initial phases to the initial phase of the target pixel in the first initial phase image to verify whether or not the corresponding pixel position having the same initial phase can be uniquely determined;
(3) finding a three-dimensional coordinate position from the uniquely determined pixel position to determine the absolute value of the initial phase of the corresponding pixel in the initial phase image; and
(4) repeating the steps (1) to (3) for the entire pixels of the initial phase image.
As a tenth aspect, in the method of the seventh aspect, the step (h) of determining an absolute value of the initial phase from an initial phase image comprises sub-steps of:
(1) finding a first set of corresponding three-dimensional coordinate position candidates from the initial phase of a target pixel in the first initial phase image, based on the projection position of the first light pattern and on the first scanning position;
(2) finding a set of pixel positions corresponding to the first set of three-dimensional coordinate positions, in the second initial phase image obtained. from the second scanning position by the first light pattern, based on the second scanning position, and comparing, in reference to respective initial phases, the initial phases to the initial phase of the target pixel in the first initial phase image to verify whether or not the corresponding pixel position having the same initial phase can be determined uniquely;
(3) finding a second set of corresponding three-dimensional coordinate position candidates from the initial phase of the uniquely determined pixel in the second initial phase image, based on the projection position of the first light pattern and the second scanning position;
(4) finding a set of pixel positions corresponding to the second set of three-dimensional coordinate positions, in the first initial phase image obtained from the first scanning position by the first light pattern, based on the first scanning position, and comparing, in reference to respective initial phases, the initial phases to the initial phase of the uniquely determined pixel in the second initial phase image to verify whether or not the corresponding pixel position having the same initial phase can be uniquely determined;
(5) finding a three-dimensional coordinate position from the pixel position uniquely determined by the two steps (2) and (4) to determine the absolute value of the initial phase of the corresponding pixel in the initial phase image; and
(6) repeating the steps (1) to (5) for the entire pixels of the initial phase image.
As an eleventh aspect, in the method of the seventh aspect, the step (h) of determining an absolute value of the initial phase from an initial phase image comprises sub-steps of:
(1) finding a set of corresponding three-dimensional coordinate position candidates from the initial phase of a target pixel in the first initial phase image, based on the projection position of the first light pattern and on the first scanning position;
(2) having reference to a set of possible phase values, from the projection position of the second light pattern and the set of three-dimensional coordinate positions, and comparing the phase values to the initial phase of the target pixel in a third initial phase image obtained from the first scanning position by the second light pattern to verify whether or not a coincident initial phase can be uniquely determined;
(3) determining an absolute value of the initial phase of the corresponding pixel in the initial phase image from the uniquely determined three-dimensional coordinate positions; and
(4) repeating the steps from (1) to (3) for the entire pixels of the first initial phase image to determine the absolute value of the initial phase.
As a twelfth aspect, in the method of the seventh aspect, the step (h) of determining an absolute value of the initial phase from an initial phase image comprises sub-steps of:
(1) a step of finding a first set of corresponding three-dimensional coordinate position candidates from the initial phase of a target pixel in the first initial phase image, based on the projection position of the first light pattern and on the first scanning position;
(2) having reference to a set of possible phase values, from the projection position of the second light pattern and the first set of three-dimensional coordinate position candidates, and comparing the phase values to the initial phase of the target pixel in a third initial Phase image obtained from the first scanning position with the second light pattern to verify whether or not a coincident initial phase can be determined uniquely;
(3) finding a second set of corresponding three-dimensional coordinate position candidates, from the initial phase of the target pixel in the third initial phase image, based on the projection position of the second light pattern and on the first scanning position;
(4) having reference to a set of possible phase values, from the projection position of the first light Pattern and the second set of three-dimensional coordinate position candidates, and comparing the phase values to the initial phase of the target pixel in the first initial phase image obtained from the first scanning position with the first light pattern to verify whether or not a coincident initial phase can be determined uniquely;
(5) determining an absolute value of the initial Phase of the corresponding pixel in the initial phase image from the three-dimensional coordinate positions uniquely determined by the steps (2) and (4); and
(6) repeating the above steps (1) to (5) for the entire pixels of the first initial phase image.
As a thirteenth aspect, in the method of the seventh aspect, the step (h) of determining an absolute value of the initial phase from an initial phase image is a combination of a plurality of steps of determining the absolute value of the initial phase of any one of 7th to 12th aspect.
As a fourteenth aspect, the method of the seventh aspect further comprises the step of projecting a light pattern having spatially striped luminance distribution from at least three different directions, respectively, on the object under measurement while phase shifting the light pattern.
As a fifteenth aspect, the method of the 7th to 14th aspects comprises the step of imaging an object under measurement from three or more different directions.
As a 16th aspect, the projected light pattern is sinusoidal.
According to a 17th aspect, there is provided a three-dimensional measurement apparatus comprising:
(a) a light projection unit for projecting a light pattern having a spatially striped luminance distribution on an objet being measured, receiving a light projection control signal, while phase shifting the light pattern responsive to the light projection control signal;
(b) first and second cameras each receiving a camera control signal as an input and formulated for scanning an image of the object under measurement responsive to the camera control signal to output image signals as a first and second camera output signal, respectively;
(c) first and second image storage memories for storing the image data captured by the first and second cameras, responsive to the first and second camera control signals, to output image data as a first and second image signal, respectively,;
(d) a phase calculation unit receiving one of the first and second image signals selected by a switch by a Phase calculation control signal, and calculating an initial phase value from pixel to pixel from a series of images of the received image signals to output calculated phase values as a first and second phase signal, respectively;
(e) first and second phase storage memories receiving the first and second Phase signals for recording the phase values of the first and second Phase signals, respectively, to output the Phase values as first and second phase signals;
(f) an absolute Phase determination unit receiving the first and second Phase signals, finding a set of corresponding three-dimensional coordinate position candidates from an initial phase of a target pixel in the first phase signal, based on positions of the light projection unit and the first camera, finding a set of pixel positions corresponding to the set of three-dimensional coordinate Position candidates in the second phase signal, based on the second camera position, and comparing in reference to the respective initial phases, the initial phases with the initial phase of the target pixel in the first initial phase signal to verify whether or not a corresponding pixel position having the same initial phase can be uniquely determined, finding a three-dimensional coordinate position from the uniquely determined pixel position, repeating the processing for determining a absolute value of the initial phase of the target pixel in the first and second Phase signals for the entire pixels of the first Phase signal, and outputting a first absolute phase signal in association with the first phase signal and outputting a second absolute phase signal in association with the second phase signal;
(g) an absolute phase conversion unit receiving the first or second absolute Phase signal and the first or second phase signal, as switched by a three-dimensional coordinate calculation control signal, to perform the processing of determining absolute phase of a pixel in the first or second phase signals, the absolute phase of the pixel being not determined yet, so that a Phase difference from an absolute phase of an ambient pixel, the absolute phase of which has a I ready been set, will become smaller, repeating the processing for the entire pixels of the first or second phase signal switched by the three-dimensional coordinate calculation signal, and for outputting the results as a first or second absolute phase conversion signal, respectively;
(h) a three-dimensional coordinate conversion unit receiving the first or second absolute phase conversion signal, as switched by the three-dimensional coordinate calculation control signal, converting the input signal into three-dimensional coordinate values based on positions of the light projection unit and the first or second camera for outputting the three-dimensional coordinate values as the first or second three-dimensional coordinate values;
(i) first and second three-dimensional coordinate storage memories receiving the first or second three-dimensional coordinate values, respectively, for recording three-dimensional coordinate values corresponding to the first and second cameras, and for outputting the three-dimensional coordinate values as first or second three-dimensional coordinate values;
(j) a three-dimensional coordinate synthesis unit receiving the and second three-dimensional coordinate signals for complementing reciprocally lacking date by way of synthesizing data to output synthesized three-dimensional coordinate signals; and
(k) a control unit outputting the light projection unit control signal, the first and second camera control signals, the phase calculation control signal and the three-dimensional coordinate calculation control signal; imaging a light pattern projected from the light projection unit while phase shifting the light pattern; and controlling input/output of Phase calculation and input/output of three-dimensional coordinate calculation.
As an 18th aspect, in the three-dimensional shape measurement apparatus of the 17th aspect, the absolute phase determination unit comprises:
(1) first means for receiving the first and second phase signals and a target pixel position signal, as input, finding a first set of corresponding three-dimensional coordinate position candidates from an initial phase of the target pixel in the first phase signal, based on positions of the light projection unit and the first camera, finding a set of pixel positions corresponding to the first set of three-dimensional coordinate position candidates in the second phase signal, based on a position of the second camera, comparing, in reference to the respective initial phases, the initial phases with the initial phase of the target pixel in the first phase signal, to verify whether or not the position of the corresponding pixel having a coincident initial phase can be determined uniquely, and for outputting the pixel position and the initial phase of the pixel on the second Phase signal can be output as a uniquely determined pixel signal;
(2) second means for receiving the first and second phase signals and the uniquely determined pixel signal, finding a second set of corresponding three-dimensional coordinate position candidates from the pixel position and the initial phase of the uniquely determined pixel signal in the second phase signal, based on the position of the light projection unit and the position of the second camera, finding a set of pixel positions corresponding to the second set of three-dimensional coordinate positions in the first Phase signal, comparing in reference to the respective initial phases, the initial Phases to the initial phase of the uniquely determined pixel in the second Phase signal to verify whether or not the position of the corresponding pixel having the same initial phase can be determined uniquely;
for finding a three-dimensional coordinate position from the pixel position in case where the corresponding pixel position has been determined uniquely by the two verifying operations; and
for determining an absolute value of the initial phases of the target pixels in the first and second phase signals and for outputting a first absolute phase signal and a second absolute phase signal in association with the first and second phase signals, respectively; and
(3) third means for sequentially scanning the entire pixels of the first Phase signal to output a position signal of the target pixel.
As a 19th aspect, the apparatus further comprises: cameras for imaging (shooting or scanning) an object from three or more different directions.
According to a 20th aspect of the present invention, here is provided a three-dimensional shape measurement apparatus comprising:
(a) first and second light projection units (A, B) receiving first and second light projection unit control signals (A,B) and for projecting first and second light patterns having spatially striped luminance distribution to an object under measurement from two different directions while phase shifting the light patterns responsive to the first and second light projection unit control signals (A,B);
(b) a camera receiving a camera control signal as an input and formulated for scanning the object under measurement responsive to the camera control signal to output an image signal as a camera output signal;
(c) first and second image storage memories (A, B) receiving the camera output signal selected through a switch as by the camera output control signal for recording a string of images captured by the camera with the first and second light patterns (A,B) to output the images as first and second Phase signals (A,B);
(d) a phase calculation unit receiving the first or second imaging signals (A or B), switched by a phase calculation control signal, as input, and calculating initial phase values from pixel to pixel from the image string to output calculated initial phase values as a phase signal;
(e) first and second phase storage memories (A, B) receiving the phase signal, allotted by the phase calculation control signal, and recording phase values calculated from the first and second image signals, to output the phase values as first and second Phase signals (A, B)
(f) an absolute phase determination unit receiving the first and second phase signals (A,B), finding a set of corresponding three-dimensional coordinate position candidates from the initial phase of the target pixel in the first phase signal (A), based on the positions of the first light projection unit (A) and the camera, having reference to a set of possible phase values from the position of the second light projection unit (B) and the set of the three-dimensional coordinate position candidates, to verify whether or not the coincident initial phase can be determined uniquely, determining absolute value of the initial Phase of the first and second light patterns from the uniquely determined three-dimensional coordinate position, and repeating the above processing of determination of the absolute values of the initial phases of the two light patterns from the uniquely determined three-dimensional coordinate position for the entire pixels of the first phase signal, to output the first and second absolute phase signals (A,B) in association with the first phase signal (A) and with the second phase signal (B), respectively;
(g) an absolute phase conversion unit receiving the first or second absolute phase signal (A or B) and the first or second phase signal (A or B), as switched by a three-dimensional coordinate calculation control signal, as input, for determining an undetermined absolute phase of a pixel in the first or second Phase signal, such that an absolute phase difference from an ambient pixel, the absolute Phase of which has already been determined, will become smaller, and repeating this processing for the entire pixels of the first or second phase signal, to output the absolute phase as a first or second absolute phase conversion signal (A or B);
(h) a three-dimensional coordinate conversion unit receiving the first or second absolute Phase conversion signal (A or B), switched by the three-dimensional coordinate calculation control signal, for converting the signal into a three-dimensional coordinate value based on the first or second light projection unit and the camera position, to output the coordinate values as first or second three-dimensional coordinate signal (A or B);
(i) first and second coordinate storage memories (A,B) receiving the first or second three-dimensional coordinate signal (A or B) as input, for recording three-dimensional coordinate values corresponding to the first and second light projection unit and for outputting the coordinate values as first or second three-dimensional coordinate signal (A or B);
(j) a three-dimensional coordinate synthesis unit receiving the first and second three-dimensional coordinate signals (A,B) as input for complementing or interpolating reciprocally lacking data for synthesis to output a synthesized signal as a synthesized three-dimensional coordinate signal; and
(k) a control unit outputting the first and second light projection unit control signals (A, B), camera control signal, camera output control signal, phase calculation control signal and the three-dimensional coordinate calculation control signal, imaging the object under measurement while switching the first and second light projection units under phase shifting, switching the camera output, controlling the input/output phase calculation and controlling the input/output of the three-dimensional coordinate calculation.
As a 21st aspect, in the apparatus of the 20th aspect, the absolute phase determining unit comprises:
(1) first means for receiving the first and second phase signals and with a target pixel position signal, as input, finding a first set of corresponding three-dimensional coordinate Position candidates from the initial phase of the target pixel in the first Phase signal, based on the positions of the first light projection unit and the camera, having reference to a set of possible phase values from the light projection unit and the set of first three-dimensional coordinate position candidates, comparing the phase values with the initial phase of the target pixel in the second phase signal, and verifying whether or not the coincident initial phase can be determined uniquely to output the pixel position of the pixel on the second Phase signal as a uniquely determined pixel signal;
(2) second means for receiving the first and second phase signals and with the uniquely determined pixel signal, as input, finding a second set of corresponding three-dimensional coordinate position candidates from the initial Phase of the target pixel in the second phase signal, based on the positions of the second light projection unit and the camera, having reference to a set of possible phase values from the light projection unit and the set of second three-dimensional coordinate position candidates, comparing the Phase values with the initial phase of the target pixel in the first phase signal, verifying whether or not the coincident initial phase can be determined uniquely, determining an absolute value of the initial phase of the two light patterns from the uniquely determined three-dimensional coordinate positions in case where the initial phases have been determined uniquely, and outputting a first absolute phase signal and a second absolute phase signal in association with the first and second phase signals, respectively; and
(3) third means for sequentially scanning the entire pixels of the first phase signal to output a target pixel position signal.
As a 21-bis aspect, the apparatus of the 20th or 21st aspect further comprises: a plurality of light projection units for projecting a light pattern having spatially striped luminance distribution on an object under measurement, from three or more different directions, while phase shifting the light pattern.
According to a 22nd aspect of the present invention, there is provided a three-dimensional shape measurement apparatus comprising:
(a) first and second light projection units A, B receiving first and second light projection means control signals A, B for projecting first and second light patterns having spatially striped luminance distribution onto an object under measurement while phase shifting the light patterns in keeping with the first and second light projection unit control signals A, B;
(b) first and second cameras A, B receiving first and second camera control signals A, B, scanning an object under measurement from two different directions in keeping with the first and second camera control signals and for outputting images of the object as first and second camera output signals A, B;
(c) first to fourth image storage memories receiving first and second camera output signals A and B, changed over by a camera output control signal, recording string of images captured by the first and second cameras with first and second light patterns, by a combination of the first light projection unit with the first camera, a combination of the second light projection unit with the first camera, a combination of the first light projection unit with the second camera and a combination of the second light projection unit with the second camera, and outputting images as first to fourth image signals A-A, A-B, B-A and B-B;
(d) phase calculating units receiving the first to fourth image signals A-A, A-B, B-A and B-B, changed over by a phase calculation control signal, to compute initial Phase values from the image string from pixel to pixel to output calculated initial phases as a phase signal;
(e) first to fourth phase storage memories receiving the phase signals changed over by the Phase calculation control signal to record phase values calculated from the first to fourth image signals A-A, A-B, B-A and B-B to output calculated Phase values as the first to fourth image signals A-A, A-B, B-A and B-B;
(f) an absolute phase determination unit receiving the first and fourth phase signals A-A and B-B as input; finding a first set of corresponding three-dimensional coordinate position candidates from an initial phase of a target pixel in the phase signal A-A obtained from the position of the first camera A by a light pattern of the first light projection unit A based on the position of the first light projection unit and the first camera A, finding a set of corresponding pixel Positions of the first three-dimensional coordinate position candidates in the phase signal B-B obtained from a position of the second camera B by the light pattern of the second light Projection unit B based on the position of the second camera B, finding a set of second three-dimensional coordinate position candidates from the initial phase of the set of the corresponding pixels, comparing the set of the first three-dimensional coordinate position candidates and the set of the second coordinate position candidates, to verify whether or not an overlapping coordinate points can be uniquely determined, determining absolute values of the initial Phases of the first and second light patterns from the uniquely determined three-dimensional coordinate positions, and repeating the above steps for the entire pixels of the phase signal A-A to output an absolute phase signal A-A in association with the phase signal A-A and to output an absolute phase signal B-B in association with the phase signal B-B;
(g) an absolute phase conversion unit receiving the absolute Phase signals A-A or B-B and with the phase signal A-A or B-B, changed over by a three-dimensional coordinate calculation control signal, as input, determining the absolute phase of pixels, the absolute phase of which in the phase signals is indeterminate, so that an absolute phase difference from the absolute value of ambient pixels, the absolute phase of which has already been determined, repeating the above steps for the entire pixels of the phase signal A-A or B-B and outputting the resulting absolute phases as absolute phase conversion signals A-A or B-B;
(h) a three-dimensional coordinate conversion unit receiving the absolute phase conversion signals A-A or B-B as input signal, changed over by three-dimensional coordinate calculation control signal, and converting the input signal into three-dimensional coordinate value based on the positions of the light projection unit A and the camera A or the light projection unit B and the camera B;
(i) first and second three-dimensional coordinate storage memories A-A, B-B receiving the three-dimensional coordinate signal A-A or B-B as input for recording three-dimensional coordinate values associated with the light projection unit A and the camera A or with the light projection unit B and the camera B. to output a three-dimensional coordinate signal A-A or B-B;
(j) a three-dimensional coordinate synthesis unit receiving the first and second three-dimensional coordinate signals A-A, B-B as input for complementing/interpolating lacking data for synthesis to produce and output the resulting signal as a synthesized three-dimensional coordinate signal; and
(k) a light projection unit/camera control unit for outputting the first and second light projection unit control signals, the first and second camera control signals, Phase calculation control signal and the three-dimensional coordinate calculation control signal for imaging a light pattern while phase shifting the light pattern to control the inputting/outputting of the Phase calculation and three-dimensional coordinate calculation.
According to a further aspect (22-bis), there is provided a three-dimensional shape measurement apparatus comprising:
(a) first and second light projection units A, B receiving first and second light projection unit control signals A, B for projecting a light pattern having spatially striped luminance distribution from two different directions on an object under measurement while Phase shifting the light pattern responsive to the first and second light projection unit control signals;
(b) first and second cameras A, B receiving first and second camera control signals A, B as input, scanning an image of an object under measurement from two different directions responsive to the first and second camera control signals and outputting the resulting signals as first and second camera output signals A, B;
(c) first to fourth image storage memories receiving the first and second camera output signals, as switched by a camera output control signal, recording a string of images captured by first and second cameras with two light patterns, by a combination of the first light projection unit with the first camera, a combination of the second light project ion unit with the first camera, a combination of the first light projection unit with the second camera and a combination of the second light projection unit with the second camera, and outputting the images as first to fourth image signals A-A, A-B, B-A and B-B;
(d) a phase calculating unit receiving the first to fourth image signals A-A, A-B, B-A and B-B, changed over by a phase calculation control signal, to compute initial phase values from the image string from pixel to pixel to output calculated initial values as a phase signal;
(e) first to fourth phase storage memories receiving the phase signals as switched over by the Phase calculation control signal to record Phase values calculated from the first to fourth image signals A-A, A-B, B-A and B-B to output the calculated phase values as the first to fourth phase signals A-A, A-B, B-A and B-B;
(f) an absolute phase conversion unit which comprises:
(f1) a first absolute phase determination unit receiving the phase signals A-A and B-A as input, determining the absolute phase value as to a pixel position, the absolute phase of which can be determined, to output the absolute phase values as absolute phase signals A-A (A-A/B-A) and absolute phase values B-A (A-A/B-A);
(f2) a second absolute phase determination unit receiving the phase signals A-B and B-B as input, determining the absolute phase value as to a pixel position, the absolute phase of which can be determined, to output the absolute phase values as absolute phase signals A-B (A-B/B-B) and absolute Phase values B-B (A-B/B-B);
(f3) a third absolute phase determination unit receiving the phase signals A-A and A-B as input, determining the absolute phase value as to a pixel position, the absolute phase of which can be determined, to output the absolute phase values as absolute phase signals A-A (A-A/A-B) and absolute phase values A-B (A-A/A-B);
(f4) a fourth absolute phase determination unit receiving the Phase signals B-A and B-B as input, determining the absolute phase value as to a pixel position, the absolute phase of which can be determined, to output the absolute phase values as absolute phase signals B-A (B-A/B-B) and absolute phase values B-B (B-A/B-B);
(f5) a fifth absolute phase determination unit receiving the phase signals A-A and B-B as input, to output an absolute phase determining pixel and its absolute phase on an image A (light pattern A) as an absolute Phase signal A-A(A-A/B-B) for a pixel position, and to output the absolute Phase determining pixel and the absolute phase on an image B (light Pattern B) as an absolute phase signal B-B(A-A/B-B), which respect to pixels the absolute phase of which can be determined; and
(f6) a sixth absolute Phase determination unit receiving the Phase signals A-B and B-A as input, to output an absolute phase determining pixel and its absolute phase on an image A (light pattern B) as an absolute phase signal A-B(A-B/B-A) for a pixel position and to output an absolute phase determining pixel and an absolute phase on an image B (light pattern A) as an absolute phase signal B-A(A-B/B-A), with respect to pixels the absolute phase of which can be determined
(f7) the absolute phase conversion unit receiving phase signals and the absolute Phase signal, both changed over by the three-dimensional coordinate calculation control signal as input, the absolute phase conversion unit converting a phase value of a pixel position in the phase signal, the absolute phase of which in the phase signal has not been acquired into an absolute phase, in reference to the absolute phase signal;
(f8) the absolute phase conversion unit receiving an input signal formulated by:
a set of a phase signal A-A and absolute phase signals A-A(A-A/B-A), A-A(A-A/A-B) and A-A(A-A/B-B),
a set of a phase signal A-B and absolute phase signals A-B(A-A/A-B), A-B(A-B/B-A) and A-B(A-A/B-B),
a set of a phase signal B-A and absolute phase signals B-A (A-A/B-A), B-A(A-B/B-A) and B-A(B-A/B-B), and/or
a set of a phase signal B-B and absolute phase signals B-B(A-A/B-B), B-B(A-B/B-B) and B-B(B-A/B-B),
the absolute phase conversion unit verifying a pixel, for which the absolute phase has been acquired, by a logical product of the entire absolute phase signals and outputting as an absolute phase conversion signal;
(g) a three-dimensional coordinate conversion unit receiving the absolute phase conversion signal and the three-dimensional coordinate calculation control signal as input,
(i) the three-dimensional coordinate conversion unit converting the absolute phase, if the absolute phase is one found from the phase signal A-A, into a three-dimensional coordinate value by parameters corresponding to a relative position between the first light projection unit A and the first camera A and to an internal structure, to output the three-dimensional coordinate value as a three-dimensional coordinate signal;
(ii) the three-dimensional coordinate conversion unit converting the absolute phase, if the absolute phase is one found from the phase signal A-B, into a three-dimensional coordinate value by parameters corresponding to a relative position between the second light projection unit and the first camera A and to the internal structure to output the three-dimensional coordinate value as a three-dimensional coordinate signal;
(iii) the three-dimensional coordinate conversion unit converting the absolute phase, if the absolute phase is one found from the phase signal B-A, into a three-dimensional coordinate value by parameters corresponding to the relative position between the second light projection unit and the second camera B and to the internal structure, to output the three-dimensional coordinate value as a three-dimensional coordinate signal; and
(iv) the three-dimensional coordinate conversion unit converting the absolute phase, if the absolute phase is one found from the phase signal B-B, into a three-dimensional coordinate value by parameters corresponding to the relative position between the second light projection unit and the second camera B and to the internal structure, to output the three-dimensional coordinate value as a three-dimensional coordinate signal;
(h) a first three-dimensional coordinate storage memory for storing three-dimensional shape information by a set of the first light projection unit A and the first camera, obtained by the three-dimensional coordinate conversion unit, for outputting a three-dimensional coordinate signal A-A;
(i) a second three-dimensional coordinate storage memory for storing three-dimensional shape information by a set of the second light project ion means A and the first camera, obtained by the three-dimensional coordinate conversion unit, for outputting a three-dimensional coordinate signal A-B;
a third three-dimensional coordinate storage memory for storing three-dimensional shape information by a set of the second light projection unit B and the first camera A, obtained by the three-dimensional coordinate conversion unit, for outputting a three-dimensional coordinate signal B-A;
(k) a fourth three-dimensional coordinate storage memory for storing three-dimensional shape information by a set of the second light project ion means B and the second camera B, obtained by the three-dimensional coordinate conversion unit, for outputting a three-dimensional coordinate signal B-B;
(l) a three-dimensional coordinate synthesis unit receiving the three-dimensional coordinate signals A-A, A-B, B-A and B-B as input for synthesizing four shape information to output synthesized information as a synthesized three-dimensional coordinate signal; and
(m) a control unit for outputting the first and second light projection unit control signals, first and second camera control signals, phase calculation control signals and the three-dimensional coordinate calculation control signals to image, scan a light pattern while phase shifting the light pattern and for controlling the inputting/outputting of the phase calculation and of the three-dimensional coordinate calculation.
According to a still further aspect (22-tris), in the apparatus of the 22-bis aspect, the first and second absolute phase determination units perform a same processing as that of the absolute phase determination unit of the 17th aspect which is decision of the absolute phase in a set of one light project ion unit and two cameras;
and wherein
the third and fourth absolute phase determination units perform a same processing as that of the absolute phase determination units of the 20th aspect which is decision of the absolute phase in a set of two light projection unit and one camera.
As the 23rd aspect, in the apparatus of the 22nd aspect, the absolute phase. determining determination unit (f) comprises first, second and third absolute phase determination means;
(1) the first absolute phase determination means receiving the phase signals A-A and B-B and a target pixel position signal, as input, finding a first set of corresponding three-dimensional coordinate position candidates from an initial phase of a target pixel in the phase signals A-A obtained from the position of the first camera A by the light pattern of the first light projection unit A, based on the positions of the first light projection unit A and the first camera A, finding a set of corresponding pixel positions of the first set of three-dimensional coordinate position candidates in the phase signal B-B obtained from the position of the second camera B, by a light pattern of the second light projection unit B, based on the position of the second camera B, finding a second set of three-dimensional coordinate position candidates from an initial phase of the set of the corresponding pixel positions, comparing the sets of the first and second three-dimensional coordinate position candidates to verify whether or not as overlapping coordinate point can be determined uniquely; and outputting the pixel position and the initial phase on the phase signal B-B as a uniquely determined pixel signal;
(2) with the second absolute phase determination means, the second absolute phase receiving the phase signals A-A and B-B and the uniquely determined pixel signals as input, finding a third set of corresponding three-dimensional coordinate position candidates from the initial phase of a pixel corresponding to the uniquely determined three dimensional coordinate position by the light pattern of the second light projection unit B based on the position of the second camera B, finding a set of corresponding pixel positions of the third set of three-dimensional coordinate position candidates on the phase signal A-A obtained from the position of the first camera A by the light pattern of the first light projecting unit A, finding a fourth set of three-dimensional coordinate Position candidates from the initial phase of the set of corresponding pixels, comparing the third and fourth sets of three-dimensional coordinate position candidates to verify whether or not overlapping coordinate points can be determined uniquely; and determining, if the overlapping coordinate points have been determined uniquely by the above two verification operations, absolute value of the initial phase of two light patterns from the uniquely determined three-dimensional coordinate position;
the second absolute phase determination means outputting the absolute phase signals A-A and B-B in association with the phase signals A-A and B-B, respectively; and
(3) the third absolute phase determination means sequentially scanning the entire pixels of the phase signal A-A to output a target pixel position signal.
As a 24th aspect, in apparatus of the 22nd aspect. the absolute Phase determination unit (f) receiving the phase signals A-A and B-A as input finds a set of corresponding three-dimensional coordinate position candidates from the initial phase of the target pixel in the Phase signal A-A, based on the position of the first light projection unit A and the first camera A, finds a set of corresponding pixel positions corresponding to the set of three-dimensional coordinate positions in the phase signal B-A obtained from the position of the second camera B from the light pattern of the first light projection unit A, based on the position of the second camera B, refers to the respective initial phases, compares the initial phase with the initial phase of the target pixel in the phase signal A-A to verify whether or not the corresponding pixel position having the same initial phase can be determined uniquely. finds a three-dimensional coordinate position by the uniquely determined pixel position and determines an absolute value of the initial phase of the target pixel; the above steps are repeated for the entire pixels of the phase signals A-A; the absolute phase determining unit outputting the absolute phase signals B-A and B-A in association with the phase signals A-A and B-B.
As a 25th aspect in the apparatus of the 22nd aspect the absolute phase determination unit (f) comprises:
(1) first means for receiving the Phase signals A-A and B-A and with a target pixel position signal as input, finds a first set of corresponding three-dimensional coordinate position candidates from the initial phase of the target pixel in the phase signal A-A, based on the position of the first light projection unit A and the first camera A, finds a set of corresponding pixel positions corresponding to the first set of three-dimensional coordinate positions in the Phase signal B-A obtained from the position of the second camera B from the light pattern of the first light projection unit A based on the position of the second camera B, refers to the respective initial phases, and compares the respective initial Phases of the target pixel in the phase signal A-A, to verify whether or not the corresponding pixel position having the same initial Phase can be uniquely determined to output the pixel position of the pixel on the phase signal B-A and the initial phase as a uniquely determined pixel signal;
(2) second means for receiving the phase signal A-A and B-A and with the uniquely determined pixel signal as input, finding a second set of corresponding three-dimensional coordinate position candidates from the initial phase of the target pixel in the Phase signal B-A, finding a set of corresponding pixel positions corresponding to the second set of three-dimensional coordinate positions in the phase signal A-A obtained from the position of the second camera A from the light pattern of the first light projection unit A based on the position of the first camera A, referring to the respective initial phases, comparing the respective initial Phases with the initial Phase of the uniquely determined pixel in the chase signal B-A, to verify whether or not the corresponding pixel position having the same initial phase can be determined uniquely, and
for finding a three-dimensional coordinate position from corresponding the pixel position if such pixel position has been determined uniquely, and for determining the absolute value of the initial phase of the target pixel in the phase signals A-A and B-A to output absolute phase signals A-A and B-A in association with the phase signals A-A and B-A; and
(3) third means for sequentially scanning the entire pixels of the phase signal A-A to output a target pixel position signal.
As a 25th aspect, in the apparatus of the 22nd aspect, the absolute phase determination unit (f) receiving the phase signals A-A and A-B finds a set of corresponding three-dimensional coordinate position candidates from the initial phase of the target pixel in the phase signal A-A, based on the position of the first light projection unit A and the first camera A, refers to a set of possible phase values from the position of the second light projection unit B and the set of three-dimensional coordinate position candidates, compares the phase values to the initial phase of the target pixel in the phase signal A-B obtained from the position of the first camera A by the light pattern of the second light projection unit B to verify whether or not the coincident initial phase can be determined uniquely, and determines an absolute value of the initial phase of the target pixel from the uniquely determined three-dimensional coordinate position; the absolute phase determining unit repeating the above steps for the entire pixels of the phase signals A-A to output an absolute phase signal A-A and A-B in association with the phase signal A-A and A-B.
As a 26th aspect of the present invention, in the apparatus of the 22nd aspect, the absolute phase determination unit (f) comprises;
(1) first means for receiving the phase signals A-A and A-B and a target pixel position signal to find a first set of corresponding three-dimensional coordinate position candidates from the initial phase of the target pixel in the phase signal A-A, based on the positions of the first light projection means A and the first camera A, referring to a set of possible phase values from the position of the second light projection unit B and the first set of three-dimensional coordinate position candidates, and comparing the phase values to the initial phase of the target pixel in the phase signal A-B obtained from the position of the first camera A by the light pattern of the second light projection unit B to verify whether or not the coincident initial phase can be determined uniquely to output the pixel position and the initial phase of the pixel on the phase signal A-B and the initial phase as a uniquely determined pixel signal;
(2) second means for receiving the phase signals A-A and A-B and the uniquely determined pixel signal to find a second set of corresponding three-dimensional coordinate position candidates from the initial phase of the target pixel in the phase signal A-B, based on the position of the second light projection means B and the first camera A, referring to a set of possible phase values from the position of the second light projection unit A and the second set of three-dimensional coordinate position candidates, comparing the phase values to the initial phase of the target pixel in the phase signal A-A obtained from the position of the first camera A by the light pattern of the first light projection unit A to verify whether or not the coincident initial phase can be determined uniquely; determining absolute value of the initial phase of the two light patterns from the three-dimensional coordinate position uniquely determined by the two verifying operations, to output the absolute phase signals A-A and A-B in association with the phase signals A-A and A-B, respectively; and
(3) third means for sequentially scanning the entire pixels of the phase signal A-A to output a target pixel position signal.
As a 27th aspect, in the apparatus of the 22nd aspect, the absolute phase determination unit (f) comprises a combination of two or more of the absolute value determination units of the initial phase selected from those shown in aspects 23rd to 28th.
In a twenty-eighth aspect of the present invention, according to the seventh to fifteenth aspects, the projected light pattern is triangular in shape.
In a twenty-ninth aspect of the present invention, according to the first to fifteenth and twenty-eighth aspects, the absolute phase values are measured in each of the three-dimensional coordinate positions, the absolute phase positions of which are previously known, and the absolute phase values are measured beforehand, which are used in calculating the corresponding three-dimensional coordinate positions from the initial phases.
In the present invention, the step of projecting a light pattern having spatially striped luminance distribution can be performed from three or more different directions, while shifting the phases. Also, in the present invention a camera or cameras may be used for imaging an object from three or more different directions.
In the three-dimensional shape measurement apparatus, the projected light pattern is sinusoidal or triangular in shape.
In the three-dimensional shape measurement apparatus, the absolute phase values are measured in each of the three-dimensional coordinate positions, the absolute coordinate positions of which are known from the outset. The absolute phase values so measured are used in calculating the three-dimensional coordinate positions from the initial phases.
According to a still further aspect, there is provided a computer program product which performs the operation as mentioned in the foregoing aspects.
Still other objects and advantages of the present invention will become readily apparent to those skilled in the art from the following detailed description, wherein only the preferred embodiments of the present invention are shown and described, simply by way of illustration of the best mode contemplated by carrying out the invention. As will be realized, the invention is capable of other and different embodiments, and its several details are capable of modifications i n various obvious respects, all without departing from the invention. Accordingly, the drawing and description are to be regarded as illustrative in nature, and not restrictive.